Not long ago I came across the problem that I wanted to know in detail how much of the allocated memory my individual lambda functions consumes.

Since memory consumption is not part of the standard Lambda metrics, I had to find an individual solution.

As each lambda execution logs the memory usage I thought about implementing a metric filter extracting this information to create a custom metric in AWS CloudWatch.

A sample metric filter was quickly found on the AWS forums (related thread).

You can test the metric filter by applying it to the log group of a lambda function like I did in the example below:

Now since verified the metric filter is actually working I only had to implement it in CloudFormation to be able to evaluate the memory consumption. It is important that a function name is defined so that the log group belonging to the Lambda function can also be created using the CloudFormation template.

You can find the template below:

Now you can find the memory consumption metric under StackName > LambdaFunctionName > Memory in AWS CloudWatch:

Read More

Two simple skeleton tempates which should be used to initialize when creating a new template from scratch. In general it is important that only the resources area is required and at least one resource must be created. All other sections are optional.

YAML skeleton

The following snippet shows the basic structure of a YAML template:

---
AWSTemplateFormatVersion: "2010-09-09"
Description: A simple skeleton
Metadata:

Parameters:

Mappings:

Conditions:

Transform:

Resources:

Outputs:

JSON skeleton

The following snippet shows the basic structure of a JSON template:

{
  "AWSTemplateFormatVersion" : "2010-09-09",

  "Description" : "A simple skeleton",

  "Metadata" : {
  },

  "Parameters" : {
  },

  "Mappings" : {
  },

  "Conditions" : {
  },

  "Transform" : {
  },

  "Resources" : {
  },

  "Outputs" : {
  }
}

Read more about the AWS CloudFormation Template Anatomy

Read More

Recently AWS released that the Amazon Simple Queue Service (SQS) is now available as a supported event source for AWS Lambda Functions. You can read the related blog post here:  https://aws.amazon.com/blogs/aws/aws-lambda-adds-amazon-simple-queue-service-to-supported-event-sources/

Since then I have seen many instructions explaining how to integrate this trigger via AWS Serverless Application Model (AWS SAM). Using this approach I noticed that when rolling out the SAM template via AWS CloudFormation a resource of type AWS::Lambda::EventSourceMapping is created. Since this resource is supported by AWS CloudFormation it should be possible to create the SQS Lambda trigger without SAM.

So I tried it successfully and got the following CloudFormation example template:

One thing to watch out for is that the lambda function timeout is not greater than the visible timeout on the queue. The solution can be tested by sending a test message via the SQS Queue using the SQS Console.

Sending a test message via the SQS Console

Check if the Lambda Function has been invoked

Read More

Sometimes in CloudFormation a Parameter requires an array, and often an array of variable size is required, determined for example by input parameters. For instance AWS::AmazonMQ::Broker were you need to define an array of SubnetIds. In which either if SINGLE_INSTANCE is selected or if ACTIVE_STANDBY_MULTI_AZ is selected, several ids must be specified.

Using the notation proposed in the documentation this cannot be achieved:

SubnetIds:
        -  !Ref PrivSubnetA
        -  !If [ DeployAmqMultiAzCondition, !Ref PrivSubnetB, ]

But the problem can be solved as follows:

SubnetIds: !If [ DeployAmqMultiAzCondition, [ !Ref PrivSubnetA, !Ref PrivSubnetB ],  [ !Ref PrivSubnetA ]]

Read More

Read More

I wanted to create an AWS CodeBuild project using AWS CloudFormation, which checks out its sources from GitHub and is triggered via GitHub Webhooks. From these sources, a Node.js application should be built using a self-created docker image stored in ECR (Elastic Container Registry).

Therefore I defined the following template:

At the first try the stack creation failed with the following error message:

No Access token found, please visit AWS CodeBuild console to connect to GitHub (Service: AWSCodeBuild; Status Code: 400; Error Code: InvalidInputException; Request ID: ab458603-6fd4-11e8-9310-ff116e0423f9)

To get rid of this error message it’s necessary to set up the AWS OAuth application to have access to your repositories.

Therefore you have to navigate to the AWS CodeBuild console, create a project and select GitHub as source provider. The project does not need to be saved, it is only important to connect to GitHub.

AWS CodeBuild GitHub

The next time I tried to deploy the CloudFormation stack, the error message did not appear and the CodeBuild project was created successfully.

CloudFormation CodeBuild

Read More

Sometimes its necessary to generate random passwords inside a CloudFormation template for instance to secure internet facing applications running on an EC2 or ECS instance. To achieve this you have the possibility to let the user of your Cloud Formation template insert passwords as parameters during the stack creation.

In the following, I will give an example of how to generate passwords in an AWS CloudFormation Template using a Node.js Lambda Function and Custom Resources.

Example Code

Description

We create a Custom CloudFormation Resource and pass a previously created Lambda function as the ServiceToken property. Now on every CloudFormation event (e.g. Create / Update / Delete) on the SampleString resource, the Lambda function will be called. The call contains a so-called ResponseUrl where the Lambda function shall response to. If you understood this procedure the template is really easy to understand. After the creation of the Custom Resource is complete you can use the data stored inside using Fn:GetAtt.

Using the Length property in the Custom Resource you can adjust the password length.

Note: The current version of the script generates a new random password if you performing a stack update which directly involves the Custom Resource (means if you change any parameter or property attached to the Custom Resource). To avoid this you could do a workaround like storing the password in an environment variable of the lambda function and resend it on an update. But normally updates on a custom resource this simple should not happen.

Usage

Inside the AWS Console go to CloudFormation and deploy the example-template.yml. After the stack creation is complete navigate to the Outputs tab and look for the generated password.

AWS Console CloudFormation

Rudimentary based on https://github.com/sophos/cloudformation-random-string example implemented in python.

Read More